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Abstract 

In this paper, we show exponential lower bounds for the class of homogeneous depth-5 
circuits over all small finite fields. More formally, we show that there is an explicit family 
{Prf : d E N} of polynomials in VNP, where P $ is of degree d inn — variables, such that 
over all finite fields F^, any homogeneous depth-5 circuit which computes P,/ must have size 
at least exp (Clq(Vd)). 

To the best of our knowledge, this is the first super-polynomial lower bound for this class 
for any field F<j ^¥ 2 . 

Our proof builds up on the ideas developed on the way to proving lower bounds for ho¬ 
mogeneous depth-4 circuits [GKKS14, FLMS14, KLSS14, KS14b] and for non-homogeneous 
depth-3 circuits over finite fields [GK98, GR00]. Our key insight is to look at the space of 
shifted partial derivatives of a polynomial as a space of functions from F” —> F^ as opposed 
to looking at them as a space of formal polynomials and builds over a tighter analysis of the 
lower bound of Kumar and Saraf [KS14b]. 


1 Introduction 

Arithmetic circuits are the most natural model to study computations of multivariate polynomials. 
These are directed acyclic graphs, with a unique sink node called the root or output gate, internal 
nodes are labelled by addition and multiplication gates * 1 , and leaves are labelled by variables or 
constants from the underlying field. The field of arithmetic circuit complexity aims at understand¬ 
ing the hardness of multivariate polynomials in terms of the size of the smallest arithmetic circuit 

* Research supported in part by NSF grant CCF-1253886 and Simons Graduate Fellowship. Part of this work done 
while an intern at MSR, New England. 

+ The research leading to these results has received funding from the European Community's Seventh Framework 
Programme (FP7/2007-2013) under grant agreement number 257575. 

1 throughout this paper, we consider circuits as having gates of unbounded fan-in. 
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computing it. One of the most important questions in this field of study is to show that there are 
families of explicit low-degree 2 polynomials that require arithmetic circuits of super-polynomial 
size (in terms of n, the number of variables). It is widely believed that the symbolic n x n perma¬ 
nent, often denoted by Perm,,, requires circuits of size exp(Q(n)) but, as of now, we do not even 
have a Cl(n 2 ) lower bound for any explicit polynomial. 

Depth Reductions 

In the absence of much progress on the question of lower bounds for general arithmetic circuits, 
a natural question to ask is if we can prove good lower bounds for nontrivial restricted classes 
of circuits. One particular class of circuits which have been widely studied with this aim are the 
class of bounded depth 3 4 arithmetic circuits. It turns out that this is not just an attempt to study a 
simpler model, but there is a formal connection between lower bounds for bounded depth circuits 
and lower bounds for general circuits. A sequence of structural results, often referred to as depth 
reduction results, show that strong enough lower bounds for bounded depth circuits implies lower 
bounds for general arithmetic circuits. 

The first depth reduction for arithmetic circuits was by Hyafil [Hya78] who showed that any 
polynomial computed by a polynomial sized arithmetic circuit can be equivalently computed 
by a circuit of depth Oflog d) and quasi-polynomial size. This was improved by Valiant, Skyum, 
Berkowitz and Rackoff [VSBR83], who showed that any n-variate degree d polynomial that can be 
computed by a circuit of size (nd)°d' can be equivalently computed by a circuit of depth 0(log d) 
and size ( nd ) 0( ^. Thus, proving super-polynomial lower bounds for Oflog d) depth circuits is 
sufficient to prove super-polynomial lower bounds for general arithmetic circuits. Agrawal and 
Vinay [AV08] further strengthened this to obtain a depth reduction to depth-4 circuits by showing 
that any n-variate degree d polynomial that can be computed by a 2 otn) sized circuit can be equiv¬ 
alently computed by homogeneous 4 depth-4 circuit of size 2°( n >. Their result was strengthened by 
Koiran [Koil2] and Tavenas [Tavl5] to show that any circuit of size s that computes an n-variate 
degree d polynomial can be computed by a homogeneous depth-4 circuit of size s O(\/rf), anc j 

in fact 

the resulting depth-4 circuits have all multiplication fan-ins bounded by O(Vd). These results 
hold over all fields. 

Over any field of characteristic zero, Gupta, Kamath, Kayal and Saptharishi [GKKS13] showed 
that any n-variate degree d polynomial computed by a size s circuit can be equivalently com¬ 
puted by a non-homogeneous depth-3 circuit of size s°(GI) Thus, these results formally show 
that proving good enough lower bounds on circuits of bounded depth is sufficient for proving 

2 where the degree is bounded by a polynomial function in the number of variables 

3 A depth k arithmetic circuit consists of k layers of alternating sum and multiplication gates with the output being 
computed by a sum gate. 

4 which means that all intermediate computations are homogeneous polynomials. Hence the degree of any interme¬ 
diate computation is bounded by the degree of the output polynomial. 
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lower bounds for general circuits. 


Lower bounds for depth-3 and depth-4 circuits 

Nisan and Wigderson [NW97] proved an exp(Q(n)) lower bound for any homogeneous depth-3 cir¬ 
cuits computing the symbolic n x n determinant Det„ by studying dimension of the partial deriva¬ 
tives of Det„ as polynomials. Grigoriev and Karpinski [GK98] and Grigoriev and Razborov [GROO] 
extended this to prove an exp(Q(n)) lower bound for non-homogenous depth-3 circuit comput¬ 
ing Det„ over any fixed finite field IF (? . Chillara and Mukhopadhyay [CM14] extended this to give 
a expiD.fd log n)) lower bound for non-homogeneous depth-3 circuits computing an h- variate 
degree d polynomial in VP. It is worth noting that there is no generic method known to convert a 
boolean lower bound for AC°[mod^] to lower bounds for arithmetic circuits over F (J (discussed in 
more detail in Section 7.1). 

The proofs of [GK98, CM14] also studied the dimension of partial derivatives of polynomial, 
but unlike the proof in [NW97], they looked at partial derivatives as functions from F” —> F i; . 
The proofs in [GK98], [GROO] and [CM14] strongly rely on the fact that we are working over 
small finite fields, and completely break down over larger fields or fields of large characteristic. 
Over fields of characteristic zero and over algebraic closure of finite fields, the question of proving 
superpolynomial lower bounds for non-homogeneous depth three circuits continues to remain 
wide open. 

Even though we had exponential lower bounds for homogeneous depth-3 circuits, the ques¬ 
tion of proving superpolynomial lower bounds for homogeneous depth-4 circuits remained open 
for more than a decade. In 2012, Kayal [Kayl2] introduced the notion of shifted partial derivatives, 
which is a generalization of the well-known notion of partial derivatives. Shifted partial deriva¬ 
tives have been very influential in a plethora of lower bounds for depth-4 circuits in the past few 
years. Gupta et. al. [GKKS14] used this measure to prove an exp(Q(y / u)) lower bound for the size 
of homogeneous depth-4 circuits with multiplication fan-ins bounded by O ( \fn ). Subsequently, 
lower bounds of exp(Q( 'fd log n )) were proved for other ;i-variate degree d polynomials com¬ 
puted by almost the same circuit class [KSS14, FLMS14, KS14a], (It is worth noting that getting a 
lower bound of exp(cu(v / dlog?i)) would have implied a super-polynomial lower bound for gen¬ 
eral circuits!) Using a more delicate variant called projected shifted partials, Kayal et. al. [KLSS14] 
and Kumar and Saraf [KS14b] proved lower bounds of exp(Q(v / dlog n )) for homogeneous depth- 
4 circuits (without any fan-in restrictions) via two very different analyses. The former was an 
analytic approach and works only over characteristic zero fields, whereas the latter was purely 
combinatorial and works over any field. These techniques have also been applied to yield lower 
bounds for non-homogeneous depth-3 circuits with bounded bottom fan-in [KS15] and homoge¬ 
neous depth-5 circuits with bounded bottom fan-in [BC15]. A continuous updated survey [Sapl5] 
contains expositions of many of the lower bounds and depth reduction results listed above. 
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The results in [KS14b] in fact show that the reduction from general arithmetic circuits to depth- 
4 circuits with support O(Vd) cannot be improved, as they give an example of a polynomial in VP 
for which any depth-4 circuits of support 0(\/d) must be of size Further, with the current 

upper-bounds for the projected shifted partials on such depth-4 circuits, the best we can hope to 
prove using this measure is an n lower bound. Hence, it might be insufficient for general 
arithmetic circuits lower bounds but it could well be the case that we might be able to prove 
stronger lower bounds for constant depth arithmetic circuits, or arithmetic formulas by variants 
of this family of measures. 

Hence, as a start, the problem of proving lower bounds for homogeneous depth five circuits, 
seems like the next natural question to explore. This already seems to introduce new challenges 
as the proofs of lower bounds for homogeneous depth-4 circuits seem to break down for homoge¬ 
neous depth-5 circuits. In this paper, we pursue this line of enquiry, and prove exponential lower 
bounds for homogeneous depth-5 circuits over small finite fields. Before stating our results, we 
first discuss prior results on this question, and the challenges involved in extending the proofs of 
lower bounds for homogeneous depth four circuits, in the next section. 

Lower bounds for depth-5 circuits 

Prior to this work, the only known lower bounds for depth-5 circuits that we are aware of are 
the results of Raz [Razl3], which show superlinear lower bounds for bounded depth circuits over 
large enough fields, the results of Kalorkoti [Kal85] which show quadratic lower bounds for arith¬ 
metic formulas and the results of Bera and Chakrabarti [BC15] and Kayal and Saha [KS15] which 
show exponential lower bounds for homogeneous depth-5 circuits if the bottom fan-in is bounded. 

Given that we have lower bounds for homogeneous depth-4 circuits, it seems natural to try and 
apply these techniques to prove lower bounds for homogeneous depth-5 circuits. Unfortunately, 
the obvious attempts to generalize the proofs in [KLSS14, KS14b] seem to fail for homogeneous 
depth-5 circuits. We now elaborate on this. 

On extending the depth-4 lower bound proofs to depth-5 circuits : To understand these is¬ 
sues, we first need a birds-eye view of the major steps in the proofs of lower bounds for depth-4 
circuits [KLSS14, KS14b]. These proofs have two major components. 

• Reduction to depth-4 circuits with bounded bottom support : In the first step, the circuit 
C and the polynomial are hit with a random restriction, in which each variable is kept alive 
independently with some small probability p. The observation is that a bottom level product 
gate in C of support (the number of distinct variable inputs) at least s survives with proba¬ 
bility at most p s . Therefore, the probability that some bottom product of support at least s 
in C survives is at most Size(C) • p s . Now, if the size of C is small (say e • 1/ p s ), then this 
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probability is quite small, so with a high probability C reduces to a homogeneous depth-4 
circuit with bounded bottom support. 

• Lower bounds for depth-4 circuits with bounded bottom support : The goal in the second 
step is to show that the polynomial obtained after random restrictions still remains hard for 
homogeneous depth-4 circuits with bottom support at most s. 

The key point in step 1 is that if Size(C) is not too large, then we can assume that with a high 
probability over the random restrictions, all the high support product gates are set to 0. This is 
where things are not quite the same for depth-5 circuits. When we express a homogeneous depth- 
5 circuit as a homogeneous depth-4 circuit by expanding the product of linear forms at level four, 
we might increase the number of monomials a lot (potentially to all possible monomials). Now, 
the random restriction step no longer works and we do not have a reduction to homogeneous 
depth-4 circuits with bounded bottom support. If the bottom fan-in of C is bounded, then this 
strategy does indeed generalize. Bera and Chakrabarti [BC15] and Kayal and Saha [KS15] show 
exponential lower bounds for such cases. 

It is not clear to us how fundamental this obstruction is, but our key insight is a strategy for 
proving lower bounds for homogeneous depth-4 circuits that avoids the random restriction step. 
Morally speaking, we do proceed by a 'reduction' from a depth-5 circuit to a depth-4 circuit, but 
the meaning of a 'reduction' here is more subtle and largely remains implicit. 

Our Contribution 

We give an exponential lower bound for homogeneous depth-5 circuits over any fixed finite field 
Fq. To the best of our understanding, this is the first such lower bound for depth-5 circuits over 
any field apart from F 2 5 . Stated precisely, we prove the following theorem. 

Theorem 1.1. There is an explicit family of polynomials { P 4 : d G IN}, with Deg (Pf) = d, in the class 
VNP such that for any finite field F ? , any homogeneous depth-5 circuit computing Pmust have size 
exp(n q (y/~d)). 

The polynomial If is from the Nisan-Wigderson family of polynomials (introduced by [KSS14], 
Definition 2.1) with carefully chosen parameters. 

Our proof also extends to non-homogeneous depth-5 circuits where the layer of multiplication 
gates closer to the output have fan-in bounded by O(Vd) (with no restriction on the fan-in of the 
other multiplication layer). 

Theorem 1.2. There is an explicit family of polynomials { P ^ : d G N}, with Deg(Prf) = d, in the 
class VNP such that for any finite field F ? , any zn^^lETE circuit computing Pmust have size 
exp (Cl q (y/d)). 

5 For F 2 , exponential lower bounds easily follow from the lower bounds of Razborov [Raz87] 
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It is worth mentioning that for characteristic zero fields, it suffices to prove an cxp(co(d ] / 3 log d) ) 
lower bound for an explicit polynomial computed by such En^^^ETE circuits to separate VP 
from VNP (by combining the depth reductions of [AV08, Koil2, Tavl5] and [GKKS13]). We elab¬ 
orate on this in Section 7.5. Such a phenomenon also happens for non-homogeneous depth three 
circuits, where over finite fields, we know quite strong lower bounds while much weaker ones 
would imply VNP 7 ^ VP over fields of characteristic zero. 

The key technical ingredient of our proof is to look at the space of shifted partial derivatives 
and the projected shifted partial derivatives of a polynomial. We study them as a space of func¬ 
tions from IF," —* F q as opposed to as a space of formal polynomials, as has been the case for 
the results obtained so far. This perspective allows us the freedom to confine our attention to the 
evaluations of the shifted partial derivatives of a polynomial on certain well chosen subsets of Fjj, 
and this turns out to be critical to our cause. This leads to a new family of complexity measures 
which could have applications to other lower bound questions as well. Our proof also involves 
a tighter analysis of the lower bound of Kumar and Saraf [KS14b] (for homogeneous depth-4 cir¬ 
cuits) which may be interesting in its own right. 

We now give an overview of our proof. 

2 An overview of the proof 


The proof would consist of the following main steps: 

1. Define a function T : F^[x] —> IN. Intuitively, we think of T(P) to be a measure of the 
complexity of P. 

2. For all homogeneous depth-5 circuits C of size at most exp (5\/d), prove an upper bound on 
T(C). 

3. For the target hard polynomial P, show that T(P) is much larger than the upper bound 
proved in step 2 . 

The complexity measure : At a high level, the proof of lower bounds in [NW97, GKKS14, KSS14, 
FLMS14, KS14a, KLSS14, KS14b] associate a linear space polynomials to every polynomial in F, ? [x] 
and use the dimension of this space over IF, ; as a measure of complexity of the polynomial. The 
mapping from polynomials to linear space of polynomials undergoes subtle changes as we go 
from the proof of lower bounds for homogeneous depth-3 circuits [NW97] to lower bounds for 
homogeneous depth-4 circuits [KLSS14, KS14b], 
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In this paper, we follow this outline and associate to every polynomial, the space of its shifted 
partial derivatives as defined in [GKKS14]. However, instead of working with this space of poly¬ 
nomials as it is, we study their evaluation vectors over a subset of F” (similar to [GK98, GROO], 
where they worked with partial derivatives of a polynomial). The key gain that we have from this 
change in outlook is that as evaluation vectors, we can choose to confine our attention to evalua¬ 
tions on certain properly chosen subsets of F”. For formal polynomials, it is not clear what should 
be the correct analog of this approximation. The necessity and the utility of this will be more clear 
as we go along. 

High rank products of linear forms : Consider a polynomial Q which is a product of t linearly 
independent linear forms L\, L 2 ,..., L r . 


T 


q=ti l < 

i=1 


It is not hard to see that 

Pr [Q(a) 7 ^ 0 ] 

acF'd V V 

In other words, products of linear forms of rank r vanish on all but a o(l) fraction of the entire 
space if r = 00 { 1). If the size of a depth-5 circuit is not too large as a function of r (say, at most 
exp(<Tr) for a small enough 6 > 0), then by a union bound, all the products of rank at least r at the 
fourth level vanish everywhere apart from a o(l) fraction of the points in F”. 

In summary, we just argued that a depth-5 circuit C over F ? of size at most exp(^r) can be 
approximated by a sub-circuit C of C which is obtained from C by dropping all products of linear 
forms of rank at least r from the bottom level. 

Low rank products of linear forms : A second simple observation (Lemma 4.3) shows that for 
every product of linear forms of rank at most r, there is a polynomial of degree at most (q — l)r, 
such that they agree at all points in F”. Thus, the circuit C is equal, as a function from F” —* F, ; 
to a depth-4 circuit C" of bottom fan-in at most (q — l)r. Moreover, the formal degree and the top 
fan-in of C" are upper bounded by the formal degree and top fan-in of C, respectively. 

Putting things together : This implies that for every homogeneous depth-5 circuit C computing 
a polynomial of degree d of size at most exp (At) for some r, there exists a depth-4 circuit C" of 
formal degree at most d and top fan-in at most exp At) such that 

Pr [C(a) A C"(a)] < o(l). 

aeFJJ 
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Therefore, a polynomial P which can be computed by C can be approximated by C" in the 
pointwise sense. Since we know lower bounds on the top fan-in of homogeneous (and low formal 
degree) depth-4 circuits with bounded bottom fan-in [GKKS14, KSS14], it seems that we only have 
a small way to go. Unfortunately, we do not quite know how to make this idea work. The key 
technical obstacle here is that it seems to be hard to say much about the partial derivatives of C by 
looking at partial derivatives of C". As a pathological case, the polynomial Hze [n] x i has a partial 
derivative span of size 2 ” but is well approximated by the 0 polynomial over F 2 . 

If we had started with a depth-3 circuit instead of a depth-5 circuit, then such a strategy is 
indeed known to work [GROO]. Observe that in this case it is enough to show that that there is 
an explicit polynomial which cannot be approximated well by a low degree polynomial over F ? . 
In [GROO], the authors show this by an adaptation of a similar result of Smolenksy [Smo87] over 
F 2 . 

A strengthening of the strategy : The key additional observation that helps us make things work 
is the fact that not only do high rank product gates at level four of C vanish almost everywhere on 
F”, but they vanish with a high multiplicity. As we show in Corollary 4.6, if the size of C is not too 
large, all the product gates of rank at least r vanish with a multiplicity Q(r) at 1 — o(l) fraction of 
points on F” 6 . 

Therefore, not only can C agree with C' almost everywhere, all the partial derivatives of C 
of order at most k = Q(r) agree with all the partial derivatives of C almost everywhere. This 
already hints at the fact that if we are to take advantage of this then we should be looking at the 
evaluation of these derivatives, since our only guarantee is about their evaluations. 

In [GK98], exponential lower bounds were proved for non-homogeneous depth-3 circuits us¬ 
ing a very similar strategy. However, adapting the method for shifted partials and projected 
shifted partials seems to be a challenge. 

In Section 4, we show that the dimension of the span of evaluation vectors of shifted partial 
derivatives of C, when restricted to a properly chosen subset S of F”, is small if the size of the 
circuit C we started with was small. 

As a final step of the proof, we show that with respect to this complexity measure, our target 
hard polynomial (from the so-called Nisan-Wigderson family, defined below) has a large complex¬ 
ity 

Definition 2.1 (Nisan-Wigderson polynomial families). Let d, m, e be arbitrary parameters with m 
being a power of a prime, and d, e < m. Since m is a power of a prime, let us identify the set [m\ with the 
field F m ofm elements. Note that since d < m, we have that [d] C F m . The Nisan-Wigderson polynomial 

6 In the rest of this discussion, we will think of r as ®(\fd) 
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with parameters d, m, e, denoted by NW^ m e is defined as 

NW d,m,e{ x ) = Yh x l,p{l) ■ ■ ■ x d,p(d) 

p{t)er m [t] 

Deg (p)<e 

That is, for every univariate polynomial p(t) S F m [t] of degree less thatn e, we add one monomial that 
encodes the ‘graph' of p on the points [d]. This is a homogeneous, multilinear polynomial of degree d over 
dm variables with exactly m e monomials. <0 

This step of the proof builds on a tighter analysis of the lower bound on the dimension of 
the span of projected shifted partial derivatives of the Nisan-Wigderson polynomials in [KS14b], We 
show that if the set S is carefully chosen, then we do not incur much loss in the dimension by going 
from looking at shifted partial derivatives as formal polynomials to looking at them as functions 
over a small subset of the finite field. We provide the details in Section 5. 

One important technicality is the dependency between various parameters involved. For our 
proof, the choice of k would be O q ( r) and would depend on q. The lower bound of [KS14b] 
would then choose specific parameters for the NW^ m e . This would mean that for every q, we get 
a different polynomial for which we show a lower bound. We remedy the order of quantifiers and 
start by fixing specific parameters for NW ( ; m/( ,. Then, depending on q, we choose a restriction of 
NW d, m ,e that would be identical to NW [( / m/ by setting some variables to 0/1. We then apply the 
[KS14b] argument for this restriction to obtain our lower bound for NW £ // m ( . which also yields a 
lower bound for NW d/me . The details are in Section 6.1. 

3 Notation 

• Throughout the paper, we shall use bold-face letters such as x to denote a set {x\, ..., x n }. 
Most of the times, the size of this set would be clear from context. We shall also abuse this 
notation to use x e to refer to the monomial xf ■ ■ ■ x‘f. 

• For an integer m > 0, we shall use [m] to denote the set {1 ,,m}. 

• We shall use the short-hand 3 x e (P) to denote 

• For a set of polynomials V shall use d =k V to denote the set of all k -th order partial derivatives 
of polynomials in V , and d- k V similarly. 

Also, x-‘ V shall refer to the set of polynomials of the form x e • P where Deg(x e ) = i and 
P G V. Similarly x- c V. 
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• For a polynomial P 6 F, [x] and for a set S C F”, we shall denote by Eva lg (P) the vector of the 
evaluation of P on points in S (in some natural predefined order like say the lexicographic 
order). For a set of vectors V, their span over F^ will be denoted by Span(V) and their 
dimension by Dim(V). 

• We shall use PL to denote the set {0,1}" CF” 

The complexity measure 

We now define the complexity measure that we shall be using to prove the lower bound. The 
measure will depend on a carefully chosen set S CFJ. 

Definition 3.1 (The complexity measure). Let k,l be some parameters and let S C IF'' . For any polyno¬ 
mial P, define T^s(P) as 

T W , S (P) := Dim{Eval s (x=^= fc (P))}. 0 


4 Complexity measure on a depth-5 circuit 

A depth-5 circuit computes a polynomial of the form 

C = LULU L abcd (4.1) 

abed 

where each L fl j, cc ? are linear polynomials. 

Definition 4.2 (Terms of a circuit, and rank). For a depth-5 circuit such as (4.1), we shall denote by 
Terms(C) the set 

Terms(C) := n hflfocd 1 

l 4 J a,b,c 

which are all products of linear polynomials computed by the bottommost product gates. 


For any term T = Yla Erf, define Rank(T) to be Dim {L^} d , which the maximum number of indepen¬ 
dent linear polynomials among the factors ofT. For a depth-5 circuit C, we shall use Rank(C) to denote 
max T6Terms(c) Rank(T). 

For a parameter r, we shall use Terms> T (C) to refer to terms T E Terms(C) with Rank(T) > r. 0 


Low rank gates are low-degree polynomials 

The following Lemma, present implicitly in [GK98, GR00], is a very useful transformation of gates 
of low-rank (and possibly large degree) when working over a finite field. 


10 


Lemma 4.3 ([GK98, GROO]). Let Q be a product of linear polynomials of rank at most r. Then, there is a 
polynomial Q of degree at most (q — 1) ■ r such that Q(a) = Q(a) for all a G F”. 

Proof. Without loss of generality, we shall assume that the rank is equal to r, as the degree upper 
bound will only be better for a smaller rank and let L\, ..., L T be linearly independent. Let 

q= n l i - n l i 

Hr] ji[r] 

Here, each linear form in the second product term is in the linear span of the linear forms {L; : 
i G [r]}, and so can be expressed as their linear combination. Therefore, Q can be expressed as a 
polynomial in {L, : i G [r]}. Let Q = f(L\, L 2 ,..., L T ). Since we are working over F fl , it follows 
that for every choice of L, and for every a G F”, we have L?(a) = L;(a). So, for every a G F^ ! , 

/(L 1 ,L 2 ,...,L T )(a) = [/(L 1 ,L 2 ,...,L t ) mod : i = 1,..., r}>](a) 

The lemma immediately follows by setting Q := f(L\, L 2 ,..., L T ) mod ( { L? — Li : i = 1,..., r}) 

□ 


High rank gates are almost always zero 

Let us assume that size(C) < 2^ //100 . We shall fix a threshold r and call all terms T with 
Rank(T) > t as “high rank terms" and the rest as "low rank terms". Under a random evalua¬ 
tion in F”, every non-zero linear polynomial takes value zero with probability 1/q. Thus, if we 
have a term that is a product of many independent linear polynomials, then with very high prob¬ 
ability many of them will be set to zero, i.e. the term will vanish with high multiplicity at most 
points. This is formalized by the following definition and the lemma after it. 

Definition 4.4 (Multiplicity at a point). For any polynomial P and a point a G F”, we shall say that a 
vanishes with multiplicity t on P ifQ( a) = 0 for all Q G d- t ~ 1 (P ). In other words, a is a root of P and 
all its derivatives up to order t — 1. 

We shall denote by Mult(P, a) the maximum t such that a vanishes on 9- t_1 (P). 0 

It is easy to see that if P is a product of linear polynomials, then a vanishes with multiplicity t 
on P if a vanishes on at least t factors of P. 

Observation 4.5. Let T = Yu=i be a term of rank at least r. Then, for every 5 > 0, 

Pr Mult(T, a) < (1 — £)- 

aeF « [ Cj 

Proof Without loss of generality, let L\,... ,L r be linearly independent. Then, the evaluations 
of Li, ...,L r at a point a G F^ ! are also linearly independent and Pr a [L ; (a) = 0] = (1/q) for 
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i = 1 

For i = 1,r, let Y, be the indicator random variable that is one if L,-( a) = 0 and zero 
otherwise. Let Y = J2ie[r] "L- Clearly, by linearity of expectations 


e[v] = E 

ieW 


r 


Since the events Y, are linearly independent, by the Chernoff Bound, we know that for every 6 > 0 


Pr 


Y < 


( 1 -*) 


q\ 



□ 


The following corollary is a simple union bound on all high-rank gates in a small circuit. 
Corollary 4.6. Let C be a depth-5 circuit over F 9 such that size(C) < 2 v/ ^ /100 . Let r = 0(\fd) so that 


exp 


8 • q 


< 2 


Yd/ 50 


Then, 


Pr 

aeF” 


3T G Terms> x (C) : Mult(T,a) < 


2 q 


< 2 


-(Yd/ 50) 


□ 


We shall set our parameter r as in the above corollary and our parameter k = r/2 q 3 


4.1 Upper bound on complexity measure 

For a circuit C of size at most 2 v/ 'C 100 , let £ refer to the "bad set" of points a such that there is some 
T G Terms> T (C) for which Mult(T, a) < k = r/2 q 3 . By the above corollary, we know that 

\£\ = 3 ■ q' 1 for some 3 = exp(—0(Vd)). 

Let S be any subset of F” \ £ that is contained in a "translate of a hypercube", that is there exists 
some c G F" such that 

Sc{c + H)\£. 

The following lemma allows us to "multilinearize" any polynomial as long as we are only inter¬ 
ested in evaluations on a translate of a hypercube. 

Lemma 4.7 (Multilinearization). Fix a translate of a hypercube c + FL. Then for every polynomial Q G 
F,j[x], there is a unique multilinear polynomial Q' such that Deg(Q / ) < Deg(Q) and Q'( a) = Q(a) for 
every a G c + TT. 

Proof. If a G c + FL, then for each i G [n] we have a, to be either c, or c, + 1. Thus, it suffices 
to replace each xj by a linear polynomial in x, that maps c, to cj and c, + 1 to (c, + l) 2 . This is 
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achieved by xf H> c? + (x, — c,) (2c, + 1). By repeated applications of this reduction, we obtain a 
multilinear polynomial Q' of degree at most Deg(Q) that agrees on all points on c + H. 

Another way to state this is by looking at Q mod X c where X c is the ideal defined by 

X c := ({*? — (<£ + (*,• —Ci)(2ci + 1)) : i = l,. 

It is easy to check that X c vanishes on c + 'H, and any Q can be reduced to a multilinear polynomial 
modulo X c . 

The uniqueness of Q' follows from the fact that no non-zero multilinear polynomial can vanish 
on all of c + XL. □ 

The main lemma of this theorem would be the following bound on the complexity measure on 
a depth-5 circuit. 

Lemma 4.8 (Upper bound on circuit). Let C be a depth-5 circuit, of formal degree at most 2d and 
size(C) < 2^ /W0 , that computes an n-variate degree d polynomial. Let r and k be chosen as above, and I 
be a parameter satisfying i + kvq < n/2. If S is any subset of IF” \ £ that is contained in a translate of a 
hyper cube, then 

r„, s (c) £ 2 ^ /1M ' ( T t ') ' G + ' poly(,,) - 

Proof. Suppose C = Ri + • • • + R s , where s < 2 v ^ /1 °o anc [ each r. i s a product of depth-3 circuits 
with Deg(R,) < 2d. Since T k//S is clearly sub-additive (i.e. T k ,e, s (f + g) < r k /, s {f) +T k / :S (g) for 
any f, g), it suffices to show that for each R,- we have 

For each such R„ define the Rf T as the polynomial obtained by "deleting" all terms T S Terms. T ( k,)- 

That is, 

if r < = nE^ R f r = n e 

a b it b:Rank(T a i,)<T 

The lemma follows from the following two claims whose proofs shall be deferred to the end of 
this section. 

Claim 4.9. For every i E [r] 

r k,e,s( R i) = r w ,s(Rp) 

Claim 4.10. For every i E [ r} 

r t ,«(RX) - ( T t +1 )'G+”kJ' poly< " ) 
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The lemma readily follows from Claim 4.9 and Claim 4.10. 


□ 


Proof of Claim 4.9. For brevity, we shall drop some indices and work with R = Qi • • • Q m . Let T E 
Terms> T (C). We shall show if R' = (Qi — T)Q 2 ■ ■ ■ Q m , then for any k- th order partial derivative 
3 x <t, 

Eval s (9 x «(R)) = Eval s (9 x «(R')). 

Indeed, consider the difference R — R 1 = T ■ Q 2 • • ■ Q m - By the chain rule, every term in 9 X « (R — R') 
is divisible by some fc'-th order partial derivative of T with k! < k. By the choice of S, we know 
that every a E S satisfies Mult(T, a) > k, and hence a vanishes on 9 : ''(T) for any T E Terms> T (C). 
Thus, it follows that Evals(3 x «(R — R')) is just the zero vector. 

Repeating this argument, we can prune away all terms in Terms> x (C) to get that Evals(3 rt (R)) = 
Eval s (3 x «(R- T )) where Deg(x a ) = k. Thus, T W/S (R) = Y k//S (R <T ). □ 

Proof of Claim 4.10. Let R- T = Qi • • • Qj, with each Q; being a ZLE circuit. Some of these Q,s 
could have degree more than r although their rank is bounded by r. Without loss of generality, 
let Qi, • • ■ , Q m be all the Q,s with Deg(Q ; ) > r, and Q m+ 1 ,..., Q d have Deg(Q,) < r. 

We shall modify the "low-degree" Q,-s by multiplying together any two of them of degree less 
than r/2. This ensures that at most one of the Q,s may have degree less than r/2 and all the Q,s 
have degree at most r. The sizes of some of the low-degree Q,s do increase in the process but this 
would not be critical as the degree of any such term is still bounded by r. The main point is that 
now we have an expression of the form 

R- T = Ql ■ ■ ■ Qm ■ Qi ■ ■ ■ Q'r 


where each Q t is a ZIIL circuit of rank at most r — 1 and Deg(Q,j > r, and all but one of the Q\ 
satisfies r > Deg(Q-) > r/2. As Deg(R- T ) < 2d, it follows that m + r < ^ + L 

As a notational convenience, for any set H let Qh := H/ch Qi- Let us look at any partial 
derivative 3 X « of order k applied on R. By the chain-rule, any such partial derivative can be written 
seen as a natural linear combination of terms. 


9 X *(R) E Span < 9 X? (Q A ) ■ 9 x7 (Qg) ■ ■ QL : 


x“ = x^ ■ x 7 , AC [m\ , 
B<Z[r] , \A\ + \B\=k 

€ Span ^ ^ ' >= T - A = W - j 

P 1 x/W 7 B C[r], \A\ + \B\=k J 


x =f 9 AR) Q Span l d x p(Q A ) ■ x- £+kT ■ Q A - Q4- 


<£+kr 


Evals(x d x *(R)) C Span Eval s (3 x/i (Q A ) ■ x- mT • Q x • QL 


x“ = x^ • x 7 , A C [; m ] , 1 
BC [r], |A| + \B\=k J 

x a = x^ • x 7 , A C [m] , 
BC [r], |A| + |B| = k 
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If we focus on the term 3 x p(Qa)/ since Qa is a product of XIIX circuits of rank at most r, we 
have that 9 x /?(Qa) is a linear combination of terms Ti ■ ■ ■ Ti a \ where each T, is a product of linear 
polynomials and has rank at most r. Using Lemma 4.3 on each of these T,s, 

Eval s (9 x /i(Q a)) G Span {Eval s (x- ( 'C 1 ^AI) j . 


Therefore, 


Eval s (x=^ x «(R)) 


C Span | Eval s (d x p (Qa) ■ x- i+kT • Q z • Q0 

C Span |Eval s (xS^+M^-Q^.q^ : 


x“ = x^ ■ x 7 , A C [m] , 1 
BC[r], |A| + |B|=* j 

x ft = x^ • x 7 , AC [m] , 1 

BC [r] , |A| + |B| =* J ’ 


Finally, Lemma 4.7 shows for every polynomial /, there is a multilinear polynomial of degree at 
most Deg(/) that agrees with / on all evaluations on a translate of a hypercube. Therefore, in the 
above span, we may assume that we are only shifting by multilinear monomials of degree t + qkr 
as this doesn't change the evaluations S C c + {0, 1}". Hence, 


Evals(x c d X “(R)) C Span 


Evals ( x 


<i+qkT n _ n / 

mult ’ «A ’ 


x a _ x/ 5 . x 7 , AC [m] , 1 
BC[r], \A\ + \B\=k J ‘ 


Therefore, using the fact that m + r < (49/r) + 1, we get the bound 

T k ,(, s {R ) := Dim | Eval s (x = ^9 =,: 


< 


f+ 1 


^ + qkT 


■ n 


where the first term corresponds to the number of choices for the subsets A and B, and the last 
two terms correspond to the number of multilinear monomials of degree at most £ + qkT. □ 


Remark 4.11. Observe that, even if the circuit C is of the form 


C = LULYl L abcd 

a b£[m] c d 

such that Size(C) < 2 v/ ^ ,/ [(l0 and m = 0(|) , then the upper bound in Lemma 4.8 continues to 
hold. In particular, the formal degree of C could be much larger than d but if the product fan-in 

7 Essentially, in the proof of Claim 4.10, we already have an expression of the form R- T = Qi • • • Q m with m = O 
and the rest of the proof proceeds as expected. 
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at level two of C is small, then 


r„, s (C) < 2^0.. ( 0 (D). (^J-polyM 0 

We later use this observation to complete the proof of Theorem 1.2 in Section 6. 

5 Lower bound for the complexity measure for an explicit polynomial 

Let £ be an arbitrary subset of IF” of size at most 5 ■ q n . We will choosing a specific set S that shall 
be a subset of a translate of the hypercube and disjoint from £. We will fix the precise definition 
of S shortly once we motivate the requirements. 

The polynomial for which was shall prove our lower bound would be from the Nisan-Wigderson 
family. We would have to set our parameters carefully but for now we shall just be intentionally 
vague and refer to the polynomial as just NW and fix parameters at a later point. 

Associated with our measure (NW) is a natural matrix that we shall call A(NW): 

The rows of A(NW) are indexed by a derivative 3 X « G d =k of order k, and a monomial 
xk of degree equal to i. The columns are indexed by all points a G S. The entry in 
(xl ] • 9 X », a) is the evaluation of xl’ : • 3 X «(NW) at the point a. 

In other words, the matrix is just the vectors Evalgfxl’ • 9 X « (NW)) listed as different rows for each 
choice of x‘ l and xk . Therefore, 

A(NW)=T m , s (NW) (5.1) 

Recall from Lemma 4.7 (multilinearization), as long as we only care about evaluations on a trans¬ 
late of a hypercube, we can assume each row is the evaluations of the multilinearization of x“ • 
9 x /s (NW). This does not change the evaluation on any point a G S C c + T-L. 

Now any such matrix of evaluations can be naturally factorized as a coefficient matrix and an 
evaluation matrix. 

C^(NW): Each row is indexed by a derivative 9 X « of order k, and a monomial xk of degree l, 
and each column is indexed by a multilinear monomial m of degree at most l + d — k over n 
variables, and the ( xk • 9 X «, m) entry is the coefficient of monomial m in the multilinearization 
of xk • 9 X *(NW) with respect to c + H (Lemma 4.7). 

Vf (S): Rows are indexed by multilinear monomials of degree at most t = i + d — k over n vari¬ 
ables, columns are indexed by a G S and (m, a) entry is the evaluation monomial m at a. 
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Clearly, A(NW) = Cy (NW) ■ V t (S). Thus if we can get a good lower bound on the ranks of the 
matrices Qt/(NW) and Vt(S) for a suitable set S, we would then be able to lower bound the rank 
of A(NW). This is formalized by the following simple linear algebraic fact. 

Lemma 5.2 (Rank of products of matrices). If A, B and C are matrices such that A = B ■ C, then 
Rank(A) > Rank(B) + Rank(C) — (■# rows of C). 

5.1 Rank of C^(NW) 

Let us focus on the matrix Qy(NW) and restrict ourselves a submatrix C( dNW) to only those 
columns whose degree is exactly t = l + d — k, and rows indexed by (x^ • 3 X «) with x- ’ being a 
multilinear monomial of degree exactly I. 

If our polynomial NW was multilinear, if we were to read off any row of CU(NW), this is 
just the list of coefficients of all multilinear monomials of (x^ • rJ x «(NW)). This is because the 
multilinearization operation in Lemma 4.7 maps any non-multilinear monomial to a multilinear 
polynomial of strictly smaller degree and hence these monomials are not included in the columns 

of cu 

The key point here is that the matrix Cj^(NW) is just the matrix of projected shifted partial deriva¬ 
tives of NW. The results of Kayal et. al [KLSS14] and Kumar and Saraf [KS14b] give a lower bound 
on the rank of this matrix for a suitable choice of the polynomial, but the lower bound of Kumar 
and Saraf [KS14b] is more relevant as it is true over any field (unlike [KLSS14] that works only 
over characteristic zero fields). 

Using a tight analysis of the argument in [KS14b], that we present in Appendix A we obtain 
the following lemma for the Nisan-Wigderson polynomial for very carefully chosen parameters. 

Lemma 5.3 (Tight analysis of [KS14b]). For every d and k = 0(\fd) there exists parameters m, e, e such 
that m = 0(d 2 ), n = md and e = © with 

m k > (l + e) 2 ^ 

/ 2 \ 

nfk = (l Te) ' poly(m) - 

For any {d, m, e, k, e} satisfying the above constraints, the polynomial N \Nd, m ,e, if — e), then over 

any field F, we have 

Rank(C M (NW d/m , e )) > Rank(C^(NW^)) > • exp(-0(log 2 d)). 
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5.2 Rank of Vt(S ) 

Let 'H<i refer to elements of {0,1}” of Hamming weight at most t. Our first step would be to 
choose our set S carefully so that we can maximize the rank of Vj(S). 

Observation 5.4. Let £ be a subset of F^' of size at most 5 ■ q n . Then for any 0 < t < n, there is a vector 
c G F" such that 

|(c ! U<t)r£\ < S*]H<t\. 

Proof. Let If (a) be the indicator function that is 1 if y G £, and 0 otherwise. Then, 

6 > I! [l £ (a)l = E [ E [lf(c + a)] . 

ye1FJ ceF” yeH< t J 

Thus, there exists some c G F” that still maintains the inequality. □ 

Our set would be S = (c + TL<t) \ £, which has the property that |S IT (c + TL<t) | > (1 — 5 )» \H<t\ 
by the above observation, and SHif = 0. 

Let Vj(S — c) be the matrix whose rows are indexed by the polynomials mix — c), where m is 
a multilinear monomials in variables x of degree at most t. The columns of V t (S — c) are indexed 
by S. We have the following observation. 

Observation 5.5. Rank(Vf(S)) = Rank(0 f (S — c)). 

Proof. For any multilinear monomial m of degree at most t, the polynomial mix — c) is multilinear 
and has degree at most t. Thus clearly, the row-space of V t (S — c) is contained in the row-space of 
Vt(S). The converse also holds trivially as the translation is invertible. □ 

We now prove our next lemma which shows a lower bound on the rank of Vf (S — c). 

Lemma 5.6. For any set S C {0,1}” C F” and any 0 < t < n, 

Rank(Vf(S-c)) = |S| 

Proof. Since S C c + T-L<t, the set S' := S — c C T-L<t- Thus the matrix Vt (S — c) is simply the matrix 
Vf(S'). For any a G {0,1}”, let m 3 refer to the 'characteristic' monomial n !:ai =i -T, and let mo = 1. 

Consider the sub-matrix of Vf(S') by restricting to rows indexed by {»; a : a G S'}. By rear¬ 
ranging the rows and columns in the increasing order of the weight of a, it is evident that the 
sub-matrix is upper-triangular with ones on the diagonal. It therefore follows that the rank of 
Vt(S') (which is just V t (S — c) ) is at least |S'| = |S|. □ 

Combining Observation 5.5 and Lemma 5.6, we have our required bound on the rank of V t (S). 
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Lemma 5.7. Let 8 be an arbitrary subset of F” of size at most 6 ■ q n . Then, there exists a set S C F” \ 8 
such that S C c + TL for some c G F^ ! for which 

Rank(V f (S)) > (l-5)-\TL< t \ = (1 - 5) ■ {# rows of V t (S)) 

Putting them together 

Lemma 5.8 (Rank bound for A(NW rf m e )). Let 8 be an arbitrary subset of F^ ! of size at most 5 ■ q n , with 
<5 = exp(—cu(log 2 d)). Then, there exists a set S C FJ^ \ 8 such that S C c + TL for some c G F ^ for 
which 

Rank(A(NW ( { / „, /( ,)) > ^ + ^ ' exp(-0(log 2 d)) 

where the parameters d, m, e, k, £ are chosen as in Lemma 5.3. 

Proof Consider the set S chosen in Lemma 5.7 (for t = £ + d — k). By Lemma 5.7, 

Rank(Vf(S)) - (# rows of V t (S)) < {-S)\H< t \ < + 

Lemma 5.3 shows that rank of Q f (NW^ m can be lower bounded by 

Ra n k (Q-^( N W dme )) > ^ + ” _ • exp(-0(log 2 d)) 

Thus, since A(NWd, m/e ) = Cy(NW,j jffl;E ) • V t (S) with t = £ + d — k, Lemma 5.2 implies that 
Rank(A(NW d/ ,„ /e )) > Rank(C w (NW d/fBje )) + Rank(Vf(S)) - (# rows of Vf(S)) 

- { e + d - k )' exp( -° (l0 « 2 

> + •exp(-0( 1 °g 2 d)) as 5 = exp(—o;(log 2 d)). □ 

Combining this with Equation 5.1, we get the following lemma. 

Lemma 5.9 (Rank bound for A(NW^„, (( ,)). Let 8 be an arbitrary subset of F^ ! of size at most 5 ■ q 11 , with 
d = exp(—cu(log 2 d)). Then, there exists a set S C F” \ 8 such that S C c + TL for some c G F” for 
which 

Rank(T WjS (NW rf/m<e )) > ^ + ^ ' exp(-Q(log 2 d)) 
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6 Wrapping up the proof 

Theorem 6.1. Let F, ; be the finite field of cardinality q. Let Cbe a depth-5 circuit of formal degree at most 
2d which computes the polynomial NW ( ^ m e with parameters as in Lemma 5.3. Then 

Size(C) > 2^ /W0 . 

Further, the same lower bound holds even ifC was a circuit of the form 

c = ri yl n Liju 

i jd[m] k l 


with m = 0(\/~d). 

Proof. We shall prove the above theorem for homogeneous depth-5 circuits. The lower bound for 
such non-homogeneous circuits would also follow directly since such circuits also have the same 
upper-bound on the complexity measure (Remark 4.11). 

Assume on the contrary that there is a circuit C computing NW^ m e with Size(C) < 2 v/ ^ 100 . 
Let r be as defined in Corollary 4.6 and let k = r/2q 3 . Let £ = £(C) be the set as defined in 
Section 4.1. We know that 

\£\ <5-q n 

for some S = exp (-O(Vd)). Let i = | (1 — e) where e = chosen as in Lemma 5.3. Since 
n = d 3 , clearly we have satisfy £ + krq < n/2. Let S C F^ ! \ £ be the set guaranteed by Lemma 5.9. 
From Lemma 5.9, we know that 

r W ,s(NW) > + •exp(-0(log 2 d)) 

This may be simplified using Lemma A.6 to 

r M ,s(NW) > • (1 + e) ld - 2k ■ exp(—0(cfe 2 )) • exp(-0(log 2 d)) 

Also, from Lemma 4.8, we know that 

r„, s (c)<2^'o°.("; 1 ).( <+ " i(tT ).poi y („) 

Again, using Lemma A.6, we get 

r w , s (C) <2^ /100 - • (”) •(l + e) 2 ‘? fcT -exp(0(-^r-e 2 ))-poly(n) 
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Since C computes NW^ m e , so it must be the case that 


2^ /100 -poly(n) > (1 + e ^( d ~ k )+( d ~ k ^ 2 c l kT ). exp(—Oq(log 2 

Since k = T/2cf, so 2qkr = r 2 / q 2 . From our choice of r in Corollary 4.6, r < 

2qkT = t 2 /cj 2 < d/36 


d)) 

2^. So 


Therefore, 


2 V ^ /100 - poly(n) > (l + e)^ • exp(-O g (log 2 d)) 

But this is a contradiction since (1 + e) id ~ k> = exp(Q( x^d log d)) by our choice of parameters. 
Therefore, the size of C is at least 2 X ^ /100 . □ 

In fact, the above proof gives more. It shows that if we have a depth-5 circuit computing 
NW d, m ,e over IF f? , then either the number of high-rank terms is at least 2 V ^ //50 or the top fan-in is 
exp(Q(v / dlogd)). 

6.1 Getting the right order of quantifiers 

In our proof so far, we first fix the field that we are working over and the parameters of our 
polynomial are then chosen based on q. Thus, as q varies, the polynomial for which we show 
the lower bound also seems to vary. The ideal scenario would be to construct a fixed polynomial 
family so that for every q we get a lower bound of exp(Q (? (v / d)). We do that now, and this would 
complete the proof of Theorem 1.1. 

We now show that this is not necessary. We would fix a family of polynomials as defined in 
Definition 2.1, and then argue that for every finite field IFq (here we think of q as 0(1)), there is a 
projection of this family, obtained by setting some of the variables to 0 or 1, such that Theorem 6.1 
holds. This would complete the proof of Theorem 1.1. 

We shall be dealing with a lot of parameters and constraints. The following is essentially the 
"zone" in which we can prove strong lower bounds (Lemma 5.3). 

Definition 6.2 (Goldilocks Zone). We shall say that parameters m,d,e,k,e zvith k = Q(\fd) and e = 
0 he in the Goldilocks Zone if they satisfy 

m k > (l + e) 2 ^) 

/ 2 \ 

nfk = (l Te) ' poly(m) - 
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0 

Recall that for Lemma 5.3, and consequently Theorem 6.1, the parameters m, d, e, k must lie in 
the Goldilocks zone. The crucial point is that this is a field dependent condition since k (and hence 
everything else) explicitly depends on q. In the next lemma, we show that we can start with a 
fixed polynomial such that for every finite field Fg of fixed size, there exists a 0,1 projection which 
lies in the Goldilocks zone. 

Lemma 6.3. Consider the NW^ /WM , polynomial with m = 0(d 2 ) and e chosen so that 

m e = 2 rf ■ poly(;n). 

Suppose k = @(\fd) and e = 0 satisfy the constraint m k > (1 + e) 2 ^ d ~ k \ Then, there ex¬ 

ists a d! E [d — 0 (\/dlogd),d] such that N\N^, me is a 0/1 projection of NW^ and the parameters 
{d', m, e, k, e} fall in the Goldilocks Zone. 

Proof. Since nf = 2 d • poly(m), m k > (1 + e) 2 ( d ~ k > and (1 + e) d = exp(0(v / dlogd)), we have 

d-k _ 

• exp(— ©(Vdlogd)). 

Since the slack in m e ~ k is just exp(0(v / dlogd)) (when compared to the desired value in Definition 6.2), 
there exists some d' E [d — 0 (Vd log d), d] such that 

m e ~ k = (^ 7 ) • poly(ra). 

Further, since m k > (1 + e) 2 ( d k \ it follows that m k > (1 + e) 2 ^' ^ as d' < d. Hence the parame¬ 
ters {d', m, e, k, e} indeed fall in the Goldilocks Zone ( Definition 6.2). 

It suffices to show that N\N^, m , e is a projection of NW^ m e . This is readily seen as setting the 
variables Xy = 1 for all i G [d — d'\ and j G [ m] yields NW^ m e up to relabelling variables. □ 

With this, we can finally prove our main theorems. 

Theorem 6.4 (Theorem 1.1 restated). Consider the polynomial NW,f zvith parameters chosen such 
that m = 0(d 2 ) and m e = 2 d ■ poly(wz). Then, for any fixed finite field Fg, any homogeneous depth-5 
circuit over F fl computing NW ^ /m/ e must have size at least 2 v/ ^ / ' 200 . 

Proof. Fix a field Fg and let k = \fd/\2cp’. 

Suppose on the contrary that there is indeed a homogeneous depth-5 circuit C computing 
N \Nd, m ,e■ Then, by Lemma 6.3, this also implies there is a projection C that computes the NWj/ ;) , v 
such that there is an d — O(\fd log d) < d' < d and there is an e = 0 ( —J for which {d r , m, e, k, e} 
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fall in the Goldilocks Zone (Definition 6.2). Now C is a circuit formal degree d < d' + 0{\fd log d) < 
2d' that computes the polynomial NW ^/ /tn , e . By Theorem 6.1, this implies that 

size(C) > size(C') > 2 V ^ 7/100 > 2 Vd/200 . □ 

The proof of this theorem also follows along the same lines. 

Theorem 6.5 ( Theorem 1.2 restated). Consider the polynomial with parameters chosen such 

that m = 0(d 2 ) and m e = 2 d ■ poly(m). Then, for any fixed finite field F^, any depth-5 circuit over F^ of 
the form 

c = EnEnv 

i jCi[m] k l 

where each is a linear polynomial and m = 0(\/d) that computes must have size at least 

2\/d/200 q 


7 Discussion 


7.1 Connections between arithmetic circuits over F^ and AC°[modt/] 

Although constant depth arithmetic circuits over Ily appear to be similar to the class AC°[mod 
q], they are surprisingly very different with respect to functions computed by them. A striking 
example, due to Agrawal, Allender and Datta [AADOO], is that arithmetic circuits over F 3 can 
"compute" both the Mod3 function, as well as the Mod2 function via 


Mod2(xi,... ,x n ) 


2 + U(l + . 


i =1 


2 


However, it is true that functions computed by arithmetic circuits over F * have strong connec¬ 
tions with AC°[modp(p* L — 1)] but unless we are working over F 2 it seems difficult lift a lower 
bound for AC°[modp] to arithmetic circuits over F p . For more on this, see [AADOO]. 

The only exception we know of is the result of Grigoriev and Razborov [GROO] where they 
lift Smolensky's [Smo87] lower bound for AC 0 [mod p] to depth-3 arithmetic circuits over F p , and 
this crucially uses the fact that depth-3 arithmetic circuits can be point-wise approximated by 
a "sparse polynomial". But in general, constant depth arithmetic circuits over F p and boolean 
circuits in AC 0 [mod p] seem to be two very different classes. 


7.2 Lower bounds for iterated matrix multiplication 

Given the results in this paper, one might wonder if the lower bounds in this paper work for a 
polynomial in VP. One natural candidate polynomial for which one might hope to show such 
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a lower bound would be the iterated matrix multiplication polynomial (IMM). It was shown in 
[KS14b] that IMM has a large complexity with respect to the measure of projected shifted partial 
derivatives. Unfortunately, the bounds in [KS14b] only show that the dimension of the space of 
projected shifted partial derivatives of the IMM (degree d in d°d) variables) are exp {—d'/d log d) 
factor close to the maximum possible value for some constant 5. This slack seems to be insufficient 
for the proofs in this paper to work as in the proof of Lemma 5.9, we relied on the fact that for the 
polynomial NW, the projected shifted partials complexity was at most a quasi-polynomial factor 
away from the largest possible. 

7.3 Finer separations for bounded depth circuits ? 

In [KS14a], it was shown that homogeneous depth-4 circuits are exponentially more powerful 
than homogeneous depth-4 circuits with bounded bottom fan-in. A natural question to ask is 
whether such separations can be shown between homogeneous depth-4 circuit and homogeneous 
depth-5 circuits. One of the first strategies to attempt for this question would be to try and show 
that there is a homogeneous depth-5 circuit such that its projected shifted partial derivative com¬ 
plexity is quite large. The results in this paper show that the measure can not to be too close 
to the largest possible value, in particular it needs to be at least a factor exp( — Cl{\fd)) away 
from the largest possible value. If this is bound is tight, then such a separation between homo¬ 
geneous depth-5 circuits and homogeneous depth-4 circuits can still be shown using projected 
shifted partial derivatives. However, it is not clear if this is the case. As mentioned before, even 
the known lower bounds on the dimension of the projected shifted partials for the IMM seem a 
factor exp (— Cl{/dlogd)) away from the largest possible value. 

7.4 The tightness of the results and relevance to VP vs VNP 

For homogeneous depth-4 circuits, we know exp (Cl(/d log cl)) lower bounds [KLSS14, KS14b] 
and any asymptotic improvement in the exponent would imply general arithmetic circuit lower 
bounds. In this sense, the lower bounds for homogeneous depth-4 circuits are tight, over all fields. 
It is natural to ask, if the bounds in this paper are tight in this sense? The answer to this question 
is far from obvious to us. In particular, it is not clear if we can use computational advantage 
of having linear forms at the bottom level of the circuit to get a better depth reduction from VP 
to homogeneous depth-4 circuits, when compared to depth reduction to homogeneous depth-4 
circuits. 

7.5 Lower bounds over fields of characteristic zero ? 

One might wonder if the techniques in this paper could be potentially adapted to work for depth- 
5 circuits over fields of characteristic zero. In the proof of Lemma 4.8, we strongly relied on the 
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fact that we are working over a fixed finite field, so it clearly seems hard to generalize over large 
fields (even when the characteristic is small). In addition to this obvious technical obstruction 
to generalizing the proof in this paper, there seems to be another reason why the proof strategy 
in this paper could be hard to replicate over fields of characteristic zero, namely, an analog of 
Theorem 1.2 over fields of characteristic zero would imply that VP f VNP. The reason is that 
over characteristic zero fields, one can obtain better depth reductions to non-homogeneous depth- 
5 circuits by combining [AV08, Koil2, Tavl5] with [GKKS13]. Although this is reasonably well 
known, we give a formal proof here for completeness. 

The following lemma is a simple generalization of the proof of depth reduction to depth-4 
circuits by Tavenas [Tavl5]. 

Lemma 7.1 (Depth reduction to homogeneous depth six circuits). Let P be a polynomial of degree d 
in poly (d) variables which can be computed by an arithmetic circuit C of size poly(d). Then , there is a 
homogeneous depth-6 circuit C which computes P and satisfies 

• Size(C) < exp (0(d 1/3 log d)), and 

• The fan-in of all the product gates in C' is bounded by 0(d 1/3 ). 

Now, we start with the circuit C as guaranteed by the lemma above, and for each of the prod¬ 
uct gates at the second level, look at its inputs. Each such input is a / circuit 

(depth-4 circuit with all product fan-ins being at most 0(d 1//3 )) of size at most exp (O^d 1 ^ log d)). 
We now apply the depth reduction to depth-3 by Gupta et al. [GKKS13] to each one of these depth- 
4 circuits. As a result, each of these depth-4 circuits get reduced to a depth-3 circuit, with at most 
a factor of exp ((^(d 1 / 3 )) blow up in size. Plugging these depth-3 circuits back into C', we obtain a 
depth-5 circuit C" such that 

• Size(C) < exp (0(d 1/3 logd)), and 

• The fan-in of all the product gates at level two of C" is bounded by 0(d l /3 ). 

Recall that the depth reduction in[GKKS13] only works over fields of characteristic zero. This 
yields the following depth reduction to non-homogeneous depth-5 circuits. 

Lemma 7.2 (Depth reduction to non-homogeneous depth-5 circuits). Let F be afield of characteristic 
zero. Let P be a polynomial of degree d in poly (d) variables over F which can be computed by an arithmetic 
circuit C of size poly (d). Then, there is a depth-5 circuit C" which computes P and satisfies 

• Size(C) < exp (0(d 1/3 logd)), and 

• The fan-in of all the product gates at level two of C is bounded by 0(d 1/3 ). 

Now, observe that an analogue of Theorem 1.2 over fields of characteristic zero, would imply 
an exp (Q(d 1//2 )) lower bound for the depth-5 circuits obtained in Lemma 7.2, and hence imply 
VP f VNP. 
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A Tight analysis of the [KS14b] lower bound 

We recall the measure of projected shifted partial derivatives that was used in [KLSS14] and [KS14b]. 

rg D (p) = Dim jmult (x=*3=*(P))} 

where mult(/) is just the polynomial / restricted to just its multilinear monomials. As mentioned 
before, this r^ D (P) is precisely Rank(C[ £ (P)) as defined in Section 5.1. 


The goal of this section would be to prove Lemma 5.3 that we restate below. 


Lemma. For every d and k = 0{\fd) there exists parameters m,e,e such that m = 0(d 2 ), n = md and 

£ = 0 {wf) With 


m k 


m 


e—k 


> (l + e) 2 ^ 

= (ifi) ■r° 1 >' (ra) ' 
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For any {d, m, e, k, e} satisfying the above constraints, the polynomial NW^ m e , if £ = | (1 — e), then over 
any field F, we have 

rg D (NW d , m , e ) > ^ + n d _ • exp(—0(log 2 d)). 

The rest of this section would just be a proof of this lemma. 

Before we proceed to the lower bound on T^ D (NW ( ; m e ), let us first show that we can indeed 
find parameters that satisfy the above constraints. Fix m to be the smallest power of 2 greater than 
d 2 to get m = 0(d 2 ). Next, we shall fix the constant cine = so that 

m k > (1+efW 

This is always possible by choosing c to be large enough as (1 + e) d ~ k = exp(O( 'fid log d)) and 
that is also the order of m k . 

Once we have done that, we shall fix e so as to ensure that 

m e ~ k = ' poly(m) 

This is always possible because choosing e = k makes the LHS < RHS and choosing e = m makes 
LHS > RHS. Hence, there must be an integer e such that LHS and RHS are within a multiplicative 
factor of m. 

All lower bounds on the dimension of shifted partial derivatives of a polynomial P was ob¬ 
tained by finding a large set of distinct leading monomials. In [KS14b], they take this approach but 
require a very careful analysis. The key difference in this setting is the following: 

If f> is the leading monomial of a polynomial P, then for any monomial 7 , we also have 
that f> • 7 is the leading monomial of 7 P. 

However, the leading monomial of mult( 7 P) could be ft • 7 for some ft fi f (as higher 
monomials could be made non-multilinear during the shift by 7 ). 

The multilinear projection makes the task of counting leading monomials much harder and 
[KS14b] come up an indirect way to count them. Throughout this discussion, let LM(/) refer to 
the leading monomial of / in some natural ordering, say the lexicographic order. 

Leading monomials after multilinear projections 

Let P the polynomial of degree d for which we are trying to lower bound For every 

monomial multilinear monomial a. of degree k, and a monomial f> G d a (P), define the set A (a, ft) 
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as 

Deg( 7 ) = £ + d — k and there is a 7 ' of degree £ 
such that 7 = LM(mult( 7 ' • 9 a (P))) = 7 ' ■ j 6 

In other words, we want the number of distinct monomials that are contributed by p, which are 
also distinct leading monomials obtained from d a (P) that are divisible by /3. We then have 


= i 7 


r PSD 

L k,e 


(P) 


> 


DL,fi 


Choice of derivatives: Instead of looking at all derivatives in d , we shall restrict ourselves to 
just a subset of derivatives. Restricting the above union to a subset A C x~ k still continues to 
remain a lower bound for T^ D (P). Keeping in mind that we are dealing with P = NW^ and 
that m k > (1 + e) 2 ( d-fc ). We shall choose A to be a set of size exactly (1 + e) 2 ( d_fc ) which consists of 
monomials of the form X\ ai • ■ • x^ ay with each a,- < m. This shall become relevant later. 


r PSD 

1 k,t 


(P) 


> 


U 

ueA 

pex =e 


(A.l) 


We shall need the following lemma from [KS14b] that is a strengthening of the standard 
Inclusion-Exclusion principle. 


Lemma A.2 (Stronger Inclusion-Exclusion [KS14b]). Let A\,..., A r be sets such that there is some 
A > 1 such that 

L\ A i nA i\ < E A- l A <l 

¥i 


Then, 



Corollary A.3. Considers sets A\,... ,A r and let Si = | A,j and S 2 = | A' fl Aj\. Then, 


UA 


> — 


Si . / Si 
— mm I 1 , — 


Estimating \\J A(cc, fi)\ via Inclusion-Exclusion 


K, fi 


a,fi 


E iM<x,p)nA(cc’,p')\ 

(« 4 i¥W) 
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Let us first address the term £ |A(a,/5)|. As mentioned earlier, it is not an easy task to get a 
good handle on the set A (a, f>) for polynomial such as NW, for any reasonable monomial ordering. 
However, [KS14b] circumvent this difficult by using an indirect approach to estimate this term. 

For any derivative a and f> G 9 a (P), define the set S(cc,j 3) as the following set of multilinear 
monomials of degree t that is disjoint from f>. 


SW) = 7: 


7 is multilinear, has 


degree t and gcd(j3, 7 ) = 1 ] 

This on the other hand is independent of any monomial ordering, and is also easy to calculate: 

'n — d + kP 


For every a, f> |S(a,j 6 )[ = f H ^ + fc Y 


Lemma A.4 ([KS14b]). For any a., 


El^W) 1 > 

p 


U SM) 

p 


Proof. Consider any 7 G S(a, f>). By definition, there is at least one non-multilinear monomial 
in 7 • 9 a (P). Thus, in particular LM(mult (7 • 9 a (P)) is non-zero and equal to some 7 • f for some 
monomial f G 9 a (P). This also implies that 7 ' = 7 • /] G A(a,/3). This yields an injective map cp 

<p:\J S (*’P) ~ {(ftV) : /3G9 a (P), 7 'G A( a ,/3)} 

J 8 


Since the size of the RHS is precisely ^ | A (a, f) |, the lemma follows. □ 

Thus, by another use of Inclusion-Exclusion on the S(a, /ij's, we get 


U A(a,/3) 

K,fi 


> El^( a ^)l - E |A(* / j6)nA(«' /j 8 , )| 

> e(eI s W)|) - e( E |s(«,j8)ns( a/j 8')| 

* V P ) * WP' 

X] |A(a,/3)nA( a ',/3')| 


Let us call the three terms in the RHS of the last equation as T\, T 2 and T 3 respectively. Since we 
know the size of each S(a, ft) exactly, the value of T\ is easily obtained. 
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Lemma A.5 ([KS14b]). 

Ti(a) := r, |S(tt/j6)| = (# mons in a deriv) ■ 

P 

We shall be simplifying such binomial coefficients very often. 

Lemma A.6. Let n and £ be parameters such that £ = 'i (1 — e) for some e = o(l). For any a, b such that 
a,b = O(yfn), 

{"-b) = (”)' 2 'Ml+er“-exp(0(6-e 2 )) 

Proof. The proof of the above lemma would repeated use the fact that n\ = (n — a)\ ■ n a ■ poly(n) 
whenever a = 0{y/n) (see [GKKS14, Lemma 3.4]). 

(”Ib) _ ( n-a)\ £\ _ (n-£)\ 

(") n! (£ — b)l (n — £ — a + b)\ 

poly 1 ^ (n- £) a 

n a (n — £) b 
(§) fl (l + e) fl (1 — e) b 
n a ’ (1 + e) b 

= 2~ a • (1 + e) a ~ zb ■ exp(0(b • e 2 )) 

□ 

Since our of parameters would be e = 0 / the bound on Ti can be simplified as 

Ti(a) = (# mons in a deriv) • + € ^j • exp(—0(log 2 d)) 

= m ^ k ■ (”) ' (~y~) •exp(-0 (log 2 d)) 

= (j) -exp(-0 (log 2 d)) 

where we used the fact that every non-zero fc-th order derivative of has exactly m e k 

monomials and our setting of parameters. 

Remark. To avoid writing this factor o/exp(0(log 2 d)), we shall use « of> or < to indicate that a factor 
exp(0(log 2 d)) is omitted. 0 

We now move on to the calculation of T 2 . This is the first place where the choice of the polynomial 
and parameters becomes crucial. 


n — d + k 
£ 
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Lemma A.7 ([KS14b]). For the polynomial P = NW^ me , ifn = mdand t = |(1 — e)fore = © 
for any a E A 


72(a) 


E |S(a / j8)nS(« / j8 , )| 


< 




2d-2A: 


Proof. Recall that S(a,f>) n S(a, ft) is just set of all multilinear monomials 7 of degree l that are 
disjoint from both f and ft. Hence, for any pair of multilinear degree (d — k ) monomials /3 f f G 
d K (P) such that Deg(gcd( J 6 /J 6')) = t, 


| s (a,/S) ns («,£')) 


/i — 2.d 2k 1 

t 


Thus, if we can count the number of pairs that agree on exactly t places, we can obtain T 2 (a). 

Note that for NW^ e , any two f>, f> G 9 (V (NWrf itn/g ) can agree on at most e — k places. Further, the 
number of pairs that agree in exactly 0 < t < e — k places is at most 


m 


e—k 


d-k 
t 


) • (m - 1 ) 


e—k—t 


as there are m e k choices for f>, and ( d . k ) choices for places where they may agree, and (m — l) e k 1 
choices for ft that agree with f> on those t places. Thus, 



/ \ / -1 I \ 2d—2k 

= m 2(e ~ k) ■ Q • (—J -0(1) if m = Q(d) □ 


Combining this with Lemma A.5 and using Inclusion-Exclusion (Corollary A.3), we get that 
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for every a E A, 


U S W) 

J 8 


> 


Ti (a) • min ^ 1 , 


Ti(«) 

r 2 (a) 


Ti(a) • min 


m e ~ k 


Ti(a) 


by our choice of parameters. Note that e needs to tailored very precisely to force the above condi¬ 
tion! If e is chosen too large or small, we get nothing from this whole exercise! 

Thus by Lemma A.4 and Lemma A.5, we get 

E l A W)l > |a|-U S M) 

aeA fi 

i 6 ea«(P) 


> |A| • 7i(a) » |A| • (”) (A.8) 


Upper bounding £ |A(a, p) (T A(a!, j6')| 

We are still left with the task of upper bounding 

T 3 = E |A(« / j8)nA(«' /j 8 , )| 

As mentioned earlier, we really do not have a good handle on the set A(a,f}), and certainly not 
on the intersection of two such sets. Once again, we shall use a proxy that is easier to estimate to 
upper bound T 3 . 

The set A(tx.,f}) n A (V, f/) consists of multilinear monomials y of degree £ -\- d — k such that 
there exists multilinear monomials y', y" of degree £ satisfying 

7 = 7 '0 = 7% 

y'P = LM(mult(ya«(P))) 
and 7 ,, / 3 / = LM(mult( 7 " 9 a /(P))) 


This in particular implies that 7 must be divisible by both (5 and j &. 
Observation A.9. if Deg(gcd(/S,/5')) = f, f/ze« 


|A(a,/3) n A(a',jS')| < 


f ti — 2d 2k H - t\ 

y £ — d + k + t J 
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Proof. Every monomial 7 E A (a, f) D A(a!, f') must be divisible by f and f>'. Since = 

2d — 2k — t, the number of choices of 7 is precisely 

/ n — (2d — 2k — t) \ _ fn-2d + 2k + t\ 

\(£ + d-k) - (2d-2k-t)J ~ \£-d+k+t) 

One needs a similar argument as in the case of T 2 to figure out how many pairs (a., f>) f (a 7 , (V) 
are there with Deg(gcd(/3, /T)) = t and sum them up accordingly. 

Lemma A.10 ([KS14b]). For the polynomial NW^ me , and n = md and l = |(1 — e) for e = © ()' 

E \AM)nA^,p>)\ < |A| 2 -(|S) 2 -(;)- 

Proof. Fix a pair of derivatives a, a.'. Let 

T 3 (a,a') := £ \A(cc,f5)nA(oc',p')l 

ped x (P) 

F&AP) 

As before, we shall first count the number of pairs of monomials ft E d a P and f E d^P such that 
gcd(j 6 /J 6 7 ) = t. Note that since a may differ from a.', we could potentially have gcd(f 1 ,( 62 ) = e. 
Once again, this is easily seen to be at most 

~.e—k ( 1 

m ■ l I • (227 — ij 


Therefore, using Observation A.9, 


T 3 (oc,oc') < Y2 m e • (wi — 1) 


\e—k—t 


t =0 
e 


d — AA f 22 — 2d T 2 A: T t 


t 


£>2 e ^ - (722 - 1 ) 


e-k- 


t /d — k\ f n\ f 1 


< 


2=0 

mf^ £ / 22 


— d + k + t 

2d-2k-t 


£ V 2 


(l + e) f 


2 2(d-k) \£ 




222 


rf-/c 


2 d—k 


t 3 


< 



□ 
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Recalling that we have chosen our parameters so that 


m 


e—k 


2 d-k 

the above equation reduces to 

+3 = 


1+e 


d-k 


and |A| = (1 + e ) 2 + k \ 


£ \A(a,p)nA(a',p')\ < JA| ■ 


Combining with (A.8), we obtain the required bound for jlj /l (a, /i) | via Inclusion-Exclusion (Corollary A.3). 


rg D (N\AW) > 


0C,p 


> 


(1 + e) 


2rf-2k 


The only thing left to observe is that by Lemma A.6, 


n 

+ d — k 


) •( 1 + e) 


2d—2k 


and that completes the proof of Lemma 5.3. 


□ 
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